Conditional Rényi Divergences and Horse Betting

A Preferred Definition of Conditional Rényi Entropy

The Rényi entropy is a generalization of Shannon entropy to a one-parameter family of entropies. Tsallis entropy too is a generalization of Shannon entropy. The measure for Tsallis entropy is non-logarithmic. After the introduction of Shannon entropy , the conditional Shannon entropy was derived and its properties became known. Also, for Tsallis entropy, the conditional entropy was introduced a...

A variational characterization of Rényi divergences

Atar, Chowdhary and Dupuis have recently exhibited a variational formula for exponential integrals of bounded measurable functions in terms of Rényi divergences. We develop a variational characterization of the Rényi divergences between two probability distributions on a measurable space in terms of relative entropies. When combined with the elementary variational formula for exponential integr...

On Rényi and Tsallis entropies and divergences for exponential families

Many common probability distributions in statistics like the Gaussian, multinomial, Beta or Gamma distributions can be studied under the unified framework of exponential families. In this paper, we prove that both Rényi and Tsallis divergences of distributions belonging to the same exponential family admit a generic closed form expression. Furthermore, we show that Rényi and Tsallis entropies c...

Stefan Berens Conditional Rényi entropy

The introduction of the Rényi entropy allowed a generalization of the Shannon entropy and unified its notion with that of other entropies. However, so far there is no generally accepted conditional version of the Rényi entropy corresponding to the one of the Shannon entropy. Different definitions proposed so far in the literature lacked central and natural properties one way or another. In this...

On the Conditional Rényi Entropy